The boom of digital entertainment, 3D printing, and virtual reality applications has led to interest in efficient and affordable ways to create 3D content. Despite the recent progress in 3D scanning for high-quality 3D content acquisition, most 3D scanning systems require expensive 3D equipment or lengthy scanning processes. Photos, on the other hand, are easy to capture and edit, and the ability to reconstruct 3D models directly from single photos could enable 3D content creation to users who do not have access to the specialized 3D equipment and lengthy scanning processes. However, reconstructing 3D shapes from single photos is a notoriously ill-posed inverse problem because photos are formed as a result of complex interactions between lighting, shape, and material properties. One approach to solve this problem is model-based techniques that use prior models to encode the shape variations of a specific object category. The shape of a human face can be well captured by such a model-based technique.
The shape of human hair, however, has not been well captured by model-based techniques because of hair's extreme variability and geometric complexity. Instead, existing single-view hair reconstruction methods have used local geometric cues such as hair occlusion and strand smoothness to reconstruct approximate hair models. Despite their adequacy for image-based rendering and editing tasks, hair models lack geometric accuracy. In addition, all model-based techniques are limited by the (usually low) dimensionality of the model and thus cannot recover characteristic fine-scale details from the photo.
Alternative methods have also failed to adequately account for the shape of hair and other detailed aspects of human portraits. Shape from Shading (SFS) methods can capture fine-scale geometric details (in the form of surface normals) for general objects from a single photo. However, existing SFS techniques cause incomplete and blurred reconstruction of hair structures because such techniques assume constant albedo, which does not apply to hair because most hairstyles have smoothly transitioned hair color. Using shape from shading techniques is also limited because their use requires knowing the lighting of the scene, which is often not known in the case of single photos.
Existing shape modeling techniques fail to adequately model hair and detailed aspects of human portrait photos and generally fail to model the hair and face in a single framework. As a result, existing techniques are inefficient and ineffective in providing 3D portrait reconstruction from single photos.